Field of the Invention
The present invention relates to an image processing technology configured to perform image processing for an image generated by image capturing.
Description of the Related Art
An image obtained by an image capturing apparatus, such as a digital camera, may deteriorate (blur) due to diffractions generated by an diaphragm (in particular, a small diaphragm aperture), even when a variety of aberrations are well corrected, such as a spherical aberration, a coma, a curvature of field, and an astigmatism of an image capturing optical system. FIG. 2 illustrates a diffraction limit curve. In FIG. 2, an abscissa axis denotes a spatial frequency, and an ordinate axis denotes a modulation transfer function (“MTF”). As an F-number increases (as a diaphragm aperture diameter reduces), a cutoff frequency shifts to a low frequency side. For instance, a Nyquist frequency of an image sensor with a pixel size of 4 μm is 125 lp (line pair)/mm. Thus, the influence of the diffraction reduces at a small F-number such as F2.8. However, the influence of the diffraction increases at a large F-number, such as F16 and F32.
Similar to a blur component caused by an aberration, a blur component caused by a diffraction is generated by imaging of spread light that derives from one point on an object and would otherwise condense at one point on an image capturing plane if there are no aberrations or no diffractions. The blur component is expressed by a point spread function (“PSF”). An optical transfer function (“OTF”) obtained by a Fourier transform of the PSF contains frequency component information of the aberration, and can be expressed by a complex number. The MTF is an absolute value or an amplitude component of the OTF, and a phase transfer function (“PTF”) is a phase component of the OTF. The MTF and PTF are frequency characteristics of the amplitude and phase components in the image deterioration caused by the aberration. The phase component is expressed as a phase angle as follows where Re(OTF) and Im (OTF) represent a real part and an imaginary part of the OTF.PTF=tan−1(Im(OTF)/Re(OTF))
Thus, the OTF of the image capturing optical system, simply referred to as an “optical system” hereinafter, deteriorates the amplitude component and the phase component of the image, and each point on the object in the deteriorated image becomes asymmetrical as in the coma.
One conventional method for correcting the deteriorated amplitude component (MTF) and deteriorated phase component (PTF) in the deteriorated image uses information of the OTF of the image capturing optical system. This method is also referred to as an image recovery or an image restoration, and a process for correcting (reducing) the deteriorated image using the OTF of the image capturing optical system will be referred to as an “image restoration process” hereinafter. One conventional method of the image restoration process is a convolution method of an image restoration filter in a real space with an input image having an inverse characteristic of the OTF, although it will be described in detail later.
A more accurate OTF of the image capturing optical system is necessary for a more effective image restoration process. The OTF can be obtained through a calculation using design value information on an image capturing optical system. The OTF can be also calculated by Fourier-transforming an intensity distribution of a captured point light source. The OTF that relates to the diffraction can be derived from a theoretically led formula.
The OTF changes according to an image capturing condition, such as an F-number and a focal length (zooming state) of an image capturing optical system, and an image height on an image capturing plane. A data amount to be stored is enormous if the OTF data is stored for each image capturing condition and for each image height. Japanese Patent Laid-Open No. (“JP”) 2012-73691 discloses a method for reducing a data amount to be stored by storing coefficient data used to reconstruct an OTF for each image pickup condition and for each image height. However, this method causes a heavy calculation burden so as to reconstruct the OTF for each image height using the coefficient data for one image.
JP 2014-150423 discloses a method for reducing a data amount and a calculation burden of an image restoration filter to be stored, by using a single image restoration filter for each F-number when the F-number is equal to or larger than a predetermined value in an image restoration process that focuses on a diffraction.
The method disclosed in JP 2014-150423 that selects the image restoration filter for each F-number needs to store data of the image restoration filter for each F-number equal to or larger than the predetermined value among F-numbers settable for the image capturing optical system. It is thus difficult to sufficiently reduce a data amount to be stored.